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Abstract — Increases in NASA mission costs are well-noted 
but not well-understood, and there is little evidence that they 
are decreasing in frequency or amount over time (Bitten and 
Freaner, 2010). The need to control spending has led to 
analysis of the causes and magnitude of historical mission 
overruns, and many program control efforts are being 
implemented to attempt to prevent or mitigate the problem 
(NPR 7120). However, cost overruns have not abated, and 
while some direct causes of increased spending may be 
obvious (requirements creep, launch delays, directed 
changes, etc.), the underlying impetus to spend past the 
original budget may be more subtle. Gaining better insight 
into the causes of cost overruns will help NASA and its 
contracting organizations to avoid 4hem. 

This paper hypothesizes that one cause of NASA mission 
cost overruns is that the availability of reserves gives project 
team members an incentive to make decisions and behave in 
ways that increase costs. We theorize that the presence of 
reserves is a contributing factor to cost overruns because it 
causes organizations to use their funds less efficiently or to 
control spending less effectively. We draw a comparison to 
the insurance industry concept of moral hazard, the 
phenomenon that the presence of insurance causes insureds 
to have more frequent and higher insurance losses, and we 
attempt to apply actuarial techniques to quantify the 
increase in the expected cost of a mission due to the 
availability of reserves. 

We create a theoretical model of reserve spending 
motivation by defining a variable ReserveSpendmg as a 
function of total reserves. This function has a positive slope; 
for every dollar of reserves available, there is a positive 
probability of spending it. Finally, the function should be 
concave down; the probability of spending each incremental 
dollar of reserves decreases progressively. We test the 
model against available NASA CADRe data by examining 
missions with reserve dollars initially available and testing 
whether they are more likely to spend those dollars, and 
whether larger levels of reserves lead to higher cost 
overruns. 

Finally, we address the question of how to prevent reserves 
from increasing mission spending without increasing cost 
risk to projects budgeted without any reserves. Is there a 
“sweet spot”? How can we derive the maximum benefit 
associated with risk reduction from reserves while 
minimizing the effects of reserve spending motivation? 
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Introduction 

Increases in NASA mission costs are well-noted but not 
well-understood, and there is little evidence that they are 
decreasing in frequency or amount over time (Bitten and 
Freaner, 2010). The need to control spending has led to 
analysis of the causes and magnitude of historical mission 
overruns, and many program control efforts are being 
implemented to attempt to prevent or mitigate the problem 
(NPR 7120). Results of these measures are only now 
becoming available for study, since the travel time from 
establishment to enforcement to outcome of such 
requirements is years. Evidence from the most recent 
mission cost data shows that the new reporting and control 
requirements have produced indifferent results at best. For 
example, Shinn et al. (2011) found that the magnitude of 
mission cost increases, measured as change in cost between 
successive milestones included in CADRe data, has no 
apparent correlation with launch year. At worst, the new 
requirements may be so cumbersome that they cost more to 
implement and enforce than they save. Some preliminary 
findings indicate that one cause of mission cost increases 
could be that costs associated with program management 
and systems engineering are estimated from historical data 
to which less onerous requirements applied. This bias causes 
current programs to underestimate effort associated with 
these activities unless historical data are adjusted 
accordingly (Shinn et al. 2011). 

As the example above demonstrates, better insight into the 
causes of cost overruns helps NASA and its contracting 
organizations to avoid them by improving cost estimating 
methods. However, cost overruns occur across all elements 
of the NASA WBS, and the data show that overruns are 
more common in spacecraft and payload than in PM and SE 
(Figure 1). While some direct causes of increased spending 
may be obvious (requirements creep, launch delays, directed 
changes, etc.), some examples of the impetus to spend past 
the original budget may be more subtle. For example. 
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suppose a mission experienced cost growth of 3 6% between 
its PDR and its launch. If it fell under the requirements of 
NPR 7120.5C, had no launch delay, and reports directed 
changes of only 10%, where d id the remaining 26% increase 
come from? 


Figure 1. Frequency of Cost Increases by WBS for 
NASA Missions in CADRe Data Set 

Several studies have been commissioned by NASA to 
evaluate the explanation of cost and schedule changes 
throughout the mission life cycle. Requirements growth, 
technical complexity, re-work, and inherent optimism are 
often cited as primary causes of cost increases. Funding 
instability can also play a factor in work authorization, 
causing projects to constantly re-plan efforts. In many of 
these cases, the “marching army” cost can increase as 
projects are pushed forward in time. Could intangible 
factors, such as political environment or knowledge of 
stakeholder intentions also play a role? It has been widely 
discussed that contractors behave differently when they 
understand how much money in total their customer may 
have for a project. Could the same phenomenon occur 
within the NASA community? 

This paper hypothesizes that one cause of NASA mission 
cost overruns is that the availability of reserves gives project 
team members an incentive to make decisions and behave in 
ways that increase costs. We theorize that the presence of 
reserves is a contributing factor to cost overruns because it 
causes organizations to use their funds less efficiently or to 
control spending less effectively. We draw a comparison to 
the economic concept of moral hazard, the phenomenon that 
the presence of insurance (or any form of contractually 
guaranteed loss-indemnification) causes insureds to have 
more frequent and higher insurance losses, (Mirrlees, 
1999). 

Reserves, cost increases, and moral 

HAZARD 

The terms reserve, management reserve, and project 
reserves are often used interchangeably in the aerospace 
environment. However, they have very different meanings 
and structure. The planning, use, and application of each 
term varies significantly. We will briefly discuss these 


terms and approach, working from the lowest level 
application to the highest level discussion. A basic tenet of 
sound project management is that a project manager should 
hold reserve to accommodate “known-unknowns.” That is, 
every project will encounter anticipated costs that cannot yet 
be specifically identified during the planning process. In the 
Earned Value Management (EVM) context, this type of 
contingency is known as Management Reserve (MR). 
Management reserve is defined as “An amount of total 
allocated budget withheld for management control purposes 
rather than designated for the accomplishment of a specific 
task or set of tasks.” (cited). The fundamental difference 
with this type of reserve is that it is held by the project and 
carved from the existing project contract budget base. It is 
wholly controlled by the project manager and can be 
allocated to specific control accounts via a baseline change 
request. The reserves and project-reserves discussed in this 
paper are the higher level holdings that are allocated in 
addition to what the project has requested (project budget). 
That is, the reserves are over and beyond the grass roots or 
parametric cost estimates generated for the completion of a 
specific mission. The NASA 7120.5D document describes 
project reserves management as “one of the most important 
functions of project management.” In this context, reserves 
are typically applied as a % or allocation over and above the 
project baseline estimate. However, these reserve amounts 
are readily known, spent, and understood by project teams. 
Further, the NASA 7120 handbook actually provides 
insights and recommendations for properly planning and 
expending reserves dollars. Our assertion is that this can 
actually exacerbate cost overruns based on the project’s 
knowledge of a “safety net.” 

Moral hazard arises when an entity changes its behavior 
because it does not bear the full consequences of the risk 
that it takes on. An example is the statistical correlation 
between automobile insurance and auto theft. Insured cars 
are more likely to be stolen than uninsured cars. One reason 
for this is that automobile owners who have insurance are 
more likely to behave carelessly about locking their cars due 
to the presence of the insurance. This phenomenon can be 
found in many different types of risk transfer arrangements, 
including banking transactions (e.g., the savings and loan 
losses of the mid-eighties), stock brokering, demand for 
medical services by insured patients, and long-term 
disability “malingering.” Many risk transfer contracts 
include risk sharing provisions that force the risk- 
advantaged entity to partake in the losses of the risk- 
disadvantaged entity to an extent commensurate with the 
expected losses associated with moral hazard. Insurance 
deductibles, medical co-pays and performance-based 
commission are all examples of risk-sharing arrangements 
to counter moral hazard. 

In the aerospace industry, cost reserves are intended, ideally, 
to absorb risks associated with estimating space mission 
costs years before launch. Estimate uncertainty is an 
accepted part of the planning process, and reserves provide 
a form of insurance against cost overruns. They protect both 
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NASA and its contracting organizations from mission 
failure caused by unexpected expenses. However, they may 
also create incentives akin to moral hazard because both the 
contracting organization and NASA have less reason to 
worry about unsustainable cost growth. Since we noted 
above that cost growth can occur for reasons within, as well 
as for reasons outside of a project’s control, reserves may 
lead to increases in spending that are not necessarily 
associated with unforeseen project risks, but rather with cost 
deltas that could otherwise be mitigated or avoided. 

Consider the following anecdotal example of a husband and 
wife planning a kitchen remodeling project. In one scenario, 
they agree that their budget for the renovation is $20,000, 
but they will set aside $5,000 (25% reserves) to cover 
unknowns. In a second scenario, their budget is also 
$20,000, but they don’t have any additional funds for 
overages. In fact, if they go over the $20,000 limit, there is 
a high likelihood that they will be unable to cover their 
mortgage payments and risk losing the house. It is clear that 
the former example has a much greater chance of overrun 
than the latter. We contend that the knowledge of reserves 
could certainly play a role in the spending of reserves. 

In order to establish whether mission cost reserves 
constitute a self-fulfilling prophecy, we need to construct a 
theory and test it against available data. A theoretical model 
of reserve spending motivation defines a variable 
ReserveSpending as a function of total reserves. 

ReserveSpending = /(Reserves') 

Equation l 

This function has a positive slope; for every dollar of 
reserves available, there is a positive probability of spending 
it. 

SReserveSpending 

— — — > 0 for all Reserves > 0 

SReserves 

Equation 2 

Finally, the function should be concave down; the 
probability of spending each incremental dollar of reserves 
decreases progressively (Equation 3). 

6 2 ReserveSpending 

— — — < 0 for all Reserves > 0 

SReserves 1 

Equation 3 

The first model requirement (Equation 1), states that the 
amount of reserves that a mission spends is a function of the 
amount of reserves that are available to the mission. The 
model does not require that reserves are the only factor 
contributing to reserve spending, as we have already shown 
above. However, we do hypothesize that the presence or 
absence of reserves affects the probability of spending 
beyond initial mission cost estimates. The second model 
requirement (Equation 2) states that the relationship 
between reserves and reserve spending is positive. Here we 


hypothesize that larger reserves lead to larger cost increases. 
The more reserve dollars are made available to cover 
mission costs, the greater the cost growth associated with 
the mission. Finally, Equation 3 hypothesizes that cost 
growth is throttled as reserve dollars are successively spent. 
As reserves begin to run out, we expect to see slow-down in 
cost growth. In other words, the first dollar of reserves is the 
most likely one to be spent on cost increases, and the last 
dollar of reserves is the least likely. 

We first tested the model against available NASA CADRe 
data. We tried to examine missions with reserve dollars 
initially available and missions to which no reserves 
applied. We tested whether reserved missions are more 
likely to experience mission cost growth between successive 
milestones, and whether larger levels of reserves lead to 
higher cost overruns. Finally, we tested whether reserve 
spending slows as reserve dollars are depleted. 

Data Collection, Normalization 
and Analysis 

We examined all available CADRe data to collect a data set 
on which to test our hypothesis. The challenge was, as 
always, getting consistent, reliable data. Several 
requirements for our data set were: 

1. Missions included require both a PDR CADRe and 
a launch CADRe. 

2. Cost data are required for total mission, spacecraft, 
payload, program management and systems 
engineering. 

3. Launch vehicle costs must be broken out. 

4. Reserves must be broken out. 

Anomalous data were excluded from our analysis. The final 
data set included 13 data points for the following missions: 


Table 1. CADRe Missions Included in this Analysis 


AIM 

CLOUDSAT 

DAWN 

GLAST 

IBEX 

LRO 

MESSENGER 

MRO 

New Horizons 

PHOENIX 

STEREO 

CALIPSO 

Spitzer 




We defined total cost overrun (which can be negative) as the 
change in mission cost between CDR and launch, excluding 
reserves and launch vehicle. Since we also collected various 
WBS elements in isolation, we defined cost increases for 
program management, systems engineering, spacecraft, and 
payload similarly, as the change in reported costs for each of 
these WBS elements between PDR and launch. Reserves 
were defined as given by the CADRe in a separate line item 
(sometimes referred to as “contingency”). 

Upon finalization of the data set, it became clear that the 
first part of our hypothesis, that reserved missions are more 
likely to experience cost overruns than unreserved missions. 
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would not be testable using available CADRe data. The 
reason for this is that all of the missions in our data set had 
reserves at PDR. We were unable to find an example of a 
mission with CADRes at both PDR and launch for which 
the cost estimate did not set aside reserves at PDR. 

The next part of our hypothesis, however, was testable. We 
examined whether mission cost increases correlate 
positively with reserve levels. A first look at total mission 
cost increase as a percentage of PDR cost estimate 
(excluding reserves and launch vehicle) against reserve 
percentage revealed no statistical significance (Figure 2). 
We encountered similarly inconclusive findings for 
spacecraft (Figure 3), payload (Figure 4), program 
management (Figure 5), and systems engineering (Figure 6) 
increases when taken as a percentage of the initial estimate. 
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Figure 2. Total Mission Cost Increase Against Reserves, 
Both as Percentages of Mission Costs Estimated at PDR, 
Exclusive of Launch Vehicle and Reserve 



Figure 3. Percentage of Increase in Spacecraft Costs 
Against Percentage Reserves 



Figure 4. Percentage of Increase in Payload Costs 
Against Percentage Reserve 



Figure 5. Percentage Program Management Cost 
Increase Against Percentage Reserve 



Figure 6. Percentage Systems Engineering Cost Increase 
Against Percentage Reserve 


We next examined whether total dollars spent bore a 
positive relationship with total dollars reserved for each of 
the cost groups. Findings demonstrated strong positive 
correlation between total mission cost increases and total 
dollars reserved, as shown in Figure 7. The same was true 
for hardware cost increases, as shown for spacecraft in 
Figure 8, and for payload in Figure 9. Figure 10 shows the 
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relationship between total cost increase of spacecraft plus 
payload as a function of total dollars of reserves. 
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Figure 7. Total Dollars of Mission Cost Increase Against 
Total Dollars Reserve 



Figure 8. Total Dollars of Spacecraft Cost Increase 
Against Total Dollars Reserved 



Figure 9. Total Dollars of Payload Cost Increase Against 
Total Dollars of Reserves 

No such strong relationship could be found between 
reserved dollars and cost increases in program management 
(Figure 1 1) and systems engineering (Figure 12). 
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Figure 10. Total Dollars of Spacecraft Plus Payload Cost 
Increase Against Total Dollars of Reserves 



Figure 11. Total Dollars of Program Management Cost 
Increase Against Total Dollars of Reserves 


Having established a strong positive relationship between 
total dollars of mission cost increase and total dollars of 
reserves, we tested the third part of our hypothesis. The 
question was whether there is evidence that cost increases 
tend to slow down as reserve dollars are used up. We 
examined cost increases as a percentage of mission reserves 
for each of the categories, total mission, spacecraft, payload, 
program management and systems engineering. 
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Figure 12. Total Dollars of Systems Engineering Cost 
Increase Against Total Dollars of Reserves 








Immediately, we encountered a problem with our 
hypothesis: within our data set, only one mission had total 
cost increase within the value of the reserves reported at 
PDR. All twelve other missions spent as much or more than 
their reserves over the PDR estimate. There did not appear 
to be evidence that mission cost increases are throttled as 
reserves run out. However, since the relationship between 
spacecraft and payload cost increases and reserve levels 
were also strong, we tested the hypothesis on these two 
categories in isolation as well. Results were surprising. 

We generated a graph showing spacecraft cost increases as a 
percentage of PDR reserves in order from smallest 
percentage increase (-11%) to largest percentage increase 
(229%) (Figure 13). The graph shows that spacecraft costs 
were increasing at an increasing rate, exactly the opposite of 
our hypothesis. However, upon closer examination of the 
graph, we discovered two groups of cost increases which 
were behaving differently. 

1. For missions whose spacecraft increases were less 
than or equal to reserves (eight total missions), cost 
increases were behaving as we expected. That is, 
they were increasing at a decreasing rate. On 
Figure 13, note that aS spacecraft costs as a 
percentage of total reserves approach 1, the bars 

> level off, showing that spending for these missions 
was slowing down. 

2. For missions whose spacecraft cost increases 
exceeded reserves (five total missions), the 
spending appears to increase at an increasing rate. 
Figure 13 shows that the data is increasing faster 
and faster for all the bars over 100%. 



Figure 13. Spacecraft Cost Increase as a Percentage of 
PDR Reserves, in order from Smallest Percentage 
Increase to Largest Percentage Increase 

To illustrate this, we separated the data into two groupings, 
shown in Figure 14 and Figure 15. 

We tested for a similar pattern with payload cost increases 
and were unable to establish a similar relationship 
conclusively. It is interesting to note, however, that payload 
cost increases also seem to show two separate groupings, 


those that experience cost increases less than total reserves, 
and those that experience cost increases greater than total 
reserves (Figure 16). 



Figure 14. Spacecraft Cost Increase as a Percentage 
PDR Reserves for all Missions Where Spacecraft 
Spending Increased Less Than Total Reserve Dollars 
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Figure 15. Spacecraft Cost Increase as a Percentage of 
PDR Reserves for all Missions Where Spacecraft 
Spending Increased More Than Total Reserve Dollars 
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Figure 16. Payload Cost Increase as a Percentage of 
PDR Reserves, in Order From Smallest Cost Increase to 
Largest Cost Increase 




Findings, Conclusions and 
Recommendations 

Although it was impossible to tell from our limited data set 
whether the existence of mission cost reserves increases the 
likelihood of mission cost increases, it was clear from this 
quantitative research that the amount of mission cost 
reserves correlates strongly with the amount of mission cost 
increases. This relationship was also strong between dollars 
of mission reserves and dollars of spacecraft and payload 
cost increase. 

Furthermore, although the third part of our hypothesis, that 
mission cost increases will slow as reserve dollars are 
successively spent (Equation 3), was disproved by our data 
set, it appears that a qualified version of this part of our 
hypothesis may yet be valid. For missions where spacecraft 
cost overruns remain within the amount of mission reserve 
dollars, overspending does, in fact, appear to be throttled as 
the spacecraft cost increases approach the total of reserves. 
However, for missions where spacecraft cost increases 
exceed total dollars of reserves, spending actually increases 
at an increasing rate, once reserve dollars are used up. 

If we confine our analysis to spacecraft cost increases, the 
question remains whether the relationship between 
overspending and reserves is causal. In order to test this, we 
will need to examine the application of reserve dollars 
across NASA institutions and investigate the chronology of 
reserve draw-down. To support the conclusion that the 
availability of reserves actually causes mission cost 
increases, we must explore whether “extra” dollars are used 
to solve project problems as they emerge, to mitigate 
mission risks, to correct for poorly estimated mission costs 
at each milestone, or to “gold-plate” a project as it nears its 
completion. 

Although this level of analysis is beyond the scope of this 
paper (or, more pertinently, it is beyond the scope of our 
available data set), we will embark on this study in the 
coming year. Some of the evidence given in this paper 
already points to the conclusion that the amount of cost 
reserves does, indeed, bear a causal relationship with the 
amount of cost overruns, particularly for spacecraft and 
payload. Assuming this is the case, how can we change 
reserves policies and practices to reduce the likelihood of 
generating incentives to overspend? 

One solution may be to alter reserve policies so that reserves 
are calculated as a function of overall mission risk, rather 
than setting reserves at a fixed percentage of total costs, 
independent of the specific characteristics that cause a 
mission to be more or less likely to use reserve dollars. 
Since reserves are intended to mitigate cost risk, they must 
be understood in terms of probability and variability. It is 
reasonable to calculate them as a whole, in terms of some 
measure of variability of the distribution of summed WBS 
costs, rather than to calculate them individually by WBS, 
thus earmarking each dollar of reserves. In light of the 


expectation that WBS elements within a total mission cost 
are often correlated with each other, it is important to note 
that the sum of the most likely outcomes (modes) of the 
individual cost distributions associated with each WBS 
element does not equal the mode of the distribution of 
summed costs (Book, 1999) (Equation 4). 

n n 

J^fmodeiWBSi) * f mode {^ WBS,) 

!=1 1=1 

Equation 4 

What is commonly referred to as the “most likely” outcome 
of the distribution of mission costs is often significantly 
lower than the true mode when correlation is taken into 
account. 

Assuming the goal is to achieve 75% percent confidence 
that sufficient funds will be available to complete a mission, 
it makes sense to calculate reserves as the difference 
between the 75* percentile of the cumulative distribution 
function of total mission costs and the most likely outcome 
(mode) of total mission costs (Equation 5). 

Reserves = F 075 (TotalMissionCosts ) 

- fmode(TotalM issionCosts) 

Equation 5 

This allows cost estimators to calculate mission costs as the 
outcome of the cost distribution that has the highest 
probability of occurring, and to quantify cost risk as the 
dollar amount required to achieve an adequate level of 
confidence in the mission’s viability. 

This approach is not unheard of. Statistical cost estimating 
techniques for space missions rely on extremely sparse data 
sets with broad definitions of similar mission characteristics. 
For example, in order to achieve a reasonably broad sample 
for this paper, we defined our data set as NASA missions 
launched since 2004 with at least two CADRe data entries. 
For this paper, as well as for any analysis used to estimate 
expected mission costs, the cost distributions are therefore 
often heavily right-skewed with very fat tails. With this type 
of data, measures of expected value are often estimated 
from the median or the mode of the distribution, rather than 
from its mean (Fleming, 2007). Adding an uncertainty 
component equal to the distance from the measure of central 
tendency to a particular percentile of the cumulative 
distribution function is a commonly used method of 
mitigating risk in insurance pricing (Miscellaneous 
insurance rate-setting paper I’ll dig up and cite). 

We have shown that reserve spending is currently (at least 
in part) a function of reserves. We have also provided strong 
evidence that reserve spending can actually be caused by the 
presence of reserves. We have offered a possible solution in 
the form of redefining reserves as a function of cost risk. 
That is, a project should directly assess Technology 
Readiness Level (TRL), complexity of missions, process 
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maturity levels, and institutional competencies. But, how 
does this align incentives so that organizations are less 
likely to use reserves? By institutionalizing this approach, 
we could examine if reserves are less likely to be spent in 
this project formulation. Limited data availability inhibits 
the complete study of whether knowledge of reserve levels 
can cause a different spending behavior. But, if a project 
(and stakeholders) were to estimate reserve necessity based 
purely on the critical elements listed above, projects could 
then generate a meaningful reserve assessment. 
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